MMFs to native and nonnative syllables show an effect of linguistic experience 
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1 Introduction 

Speech perception involves mapping variable 
acoustic signals onto a limited set of mental 
representations of speech sounds. Early work 
revealed that listeners showed increased sensitivity 
to acoustic differences for cross-category synthetic 
tokens when compared to within-category tokens 
even though the physical difference was equated [1]. 
This “categorical perception” phenomenon was also 
observed in infants as young as 1 month olds and 
nonhuman mammals [2,3], While natural phonetic 
boundaries may initially be inherent in the 
mammalian auditory system, auditory sensitivities 
to speech stimuli change substantially with linguistic 
exposure early in life [4,5], As adults, we are 
especially sensitive to our native language phonetic 
contrasts, and it is often difficult for us to 
discriminate normative speech sounds. 

The overwhelming behavioral evidence that speech 
perception is altered by linguistic experience has 
led to studies exploring the underlying neural 
mechanisms of speech representations. Recent work 
using event related potentials (ERP) and 
magnetoencephalography (MEG) indicates that 
certain neural measures, such as mismatch 
negativity (MMN) and mismatch field (MMF), 
reflect not only pre-attentive sensory detection of 
acoustic deviation from a standard but also a higher 
level of processing for speech stimuli that involves 
language-specific representations [6,7,8], These 
studies have shown that, given equal acoustic 
differences in the stimuli, the mismatch responses 
for normative speech contrasts are significantly 
diminished. On the other hand, studies also show 
that the MMN can be elicited for within-category 
stimuli in the absence of good behavioral 
discrimination [9,10,11 ]. 

Our purpose was to use both native and normative 
contrasts and two language groups to examine 
whether the MMN/MMF measures is a robust index 
of the effects of linguistic experience. Previous 
MMN/MMF experiments have examined vowel 
categories and stop consonant categories with 


contrasts in place of articulation and voicing. 
Additional tests using consonant categories in 
manner of articulation are necessary to evaluate the 
generality of the results. The behavioral and 
neurophysiologic experiments in this study also 
provide a basis for an ongoing study looking at 
brain plasticity in learning normative phonemes. 

The data reported here are preliminary results 
obtained from Japanese and American subjects on 
two contrasts, /ba/ vs. /wa/, and /la/ vs. /ra/. We 
predicted that in Japanese subjects the native /ba/- 
/wa/ contrast would produce large MMF responses 
whereas the non-native contrast /la/-/ra/ would 
produce either a smaller MMF, or no MMF as 
previously reported [12]. We further predicted that 
comparisons between Japanese and American 
subjects would show cross-language differences for 
the /la/-/ra/ contrast, but not the /ba/-/wa/ contrast. 

2 Methods 

2.1 Behavioral experiments 

The subjects were ten native speakers of Japanese (3 
females, 7 males, age range: 21-24, mean 22.3) and 
ten native speakers of American English (5 females, 
5 males, age range: 20-30, mean 23.6). Japanese 
subjects were all college students in Japan who 
received English instruction in reading and writing 
in the middle school and college. The American 
subjects were all monolingual students at the 
University of Washington. All subjects were right- 
handed with no history of speech and hearing 
disorders. 

The stimuli were two synthesized continua, /ba-wa/ 
and /ra-la/. Figure 1 shows the spectrograms of 
endpoints in the two continua. Specific parameters 
were adopted from previous studies [13, 14], The 
/ba-wa/ continuum consisted of nine syllables with a 
fixed initial 20 ms of prevoicing in the first formant. 
All acoustic parameters were identical except for 
the critical one distinguishing the syllables, the 
duration of the FI and F2 transitions, which varied 
from 16 to 48 ms in 4-ms steps. The FI started at 
234 Hz with a linear rising slope to a steady state of 



769 Hz. F2 started at 616 Hz and rose linearly to a 
steady state of 1,232 Hz. F3 was constant at 2,862 
Hz. The stimulus duration was 100 ms. The /ra-la/ 
continuum had eleven syllables, each 400 ms in 
duration. The syllables started with a 155 ms steady 
formant structure , followed by a 35 ms FI transition 
from 351 to 769 Hz, a 35 ms F2 transition from 
1003 Hz to 1221 Hz, and a 100 ms F3 transition to a 
steady of 2973 Hz for the vowel /a/. The F3 
transition slopes had starting frequencies between 
1325 and 3649 Hz, spaced equally in mels at eleven 
levels. All syllables were resampled at 48 kHz, at 
16-bit resolution, to accommodate the MEG 
stimulator. 
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Figure 1: Spectrograms for the endpoint stimuli. 

A phonetic identification task was implemented via 
computer. Identification tasks require knowledge of 
linguistic categories and can well predict the 
discrimination of speech sounds; discrimination is 
easy when sounds receive different phonetic labels 
and difficult when sounds that receive the same 
label [13], Subjects completed the tests in an 
acoustically treated booth. For each continuum, 
there was a practice block of 20 trials followed by a 
testing session of 40 trials for each stimulus. The 
stimuli were randomly presented to the right-ear 
Telephonies headset at the output of 80dB SPL. 

2.2 MEG experiments 

The MEG experiments involved the same ten 
Japanese subjects. Given the accuracy of MEG data, 
which is meaningful on the single-subject level [15], 
one American female was used as a control. 
Subjects were instructed to read a self-chosen book 
without attending to the stimuli in the oddball 
paradigm. The endpoint stimuli in the two continua 
were used as the standard and deviant. Stimuli were 
monoaurally delivered to the right ear using the 
earpiece which was connected to a plastic tube. The 
stimulator output level was 80 dB SPL. There were 
two blocks of stimuli with the standard and the 
deviant reversed in the second block. Block 
sequences were counterbalanced across subjects. 
The deviant was presented with 0.15 probability 
with at least two intervening standards. Interstimulus 
intervals were randomized between 800 and 1200 
ms. To reduce fatigue and habituation, a ten-minute 
break was inserted between blocks. After the break, 


head positioning data were refitted on four coils 
pasted on the scalp with goodness of fit over 98% 
and head origin deviations in the range of 0 ~ 
3.0 mm. 

The data were collected using the Neuromag 122- 
channel whole-head SQUID gradiometer housed in 
a four-layered magnetically shielded room at NTT 
Communications Science Laboratories in Japan. The 
analog filter was 0.01 -100 Hz, and the sampling 
frequency was 497 Hz. Epochs with MEG > 3000 
fT/crn or EOG >150 pV due to artifacts such as eye 
movements monitored by horizontal and vertical 
bipolar EOG channels were rejected online. At least 
80 epochs were averaged for the deviant and for the 
standard presented immediately before the deviant. 
The data were digitally filtered at 0.8 ~ 40 Hz 
offline. The analysis time was -100 ~ 800 ms. Nlm 
was individually determined using a subset of 22 
channels at the frontal and temporal lobes of each 
hemisphere. The MMF peak was determined from 
difference waves using a time window of 200 ms 
after Nlm. Equivalent current dipole (ECD) was 
estimated by the revised local channel method, 
which assumed single current dipole in a spherical 
head model [16]. 

3 Results 

3.1 Behavioral identification functions 

The two subject groups showed similar performance 
for the /ba-wa/ continuum and striking differences 
for the /ra-la/ continuum (Figure 2). 

» 100% 

I 80% 

1 15 60% 

~ 5 40% 

| 20% 

£ 0% 

1 2 3 4 5 6 7 8! 

/ba-wa/ Stimulus Continuum 



ES 100% 

I 80% 



| 20 % 


8 o% 

123456789 1011 
/ra-la/ Stimulus Continuum 



Figure 2: Japanese and American subjects’ average 
identification functions for the two continua. 


Unlike the other three identification functions 
indicative of categorical perception, the Japanese 











/ra-la/ identification curve was continuous and 
limited to the 29.5% ~ 78.5% range. There were no 
significant between-group differences for the /ba- 
wa/ stimuli but significant differences on every /ra- 
la/ stimulus except No. 5 which happens to be at the 
American /1-r/ phonetic boundary (two-tailed 
Kolmogorov-Smimov test, p < .05). In particular, 
comparisons on the endpoint stimuli were as 
follows: /ba/: Z = .45, p =.99; /wa/: Z = .67, p = .76; 
/ra/: Z = 1.79, p < .01; /la/: Z = 2.24, p < .001. 

3.2 Neuromagnetic data 

The MEG data showed clear Nlm patterns in the 
channels over the left and right temporal lobes 
(Figure 3). In the /ba-wa/ experiment for Japanese 
subjects, a significant main effect was found for 
stimulus in a repeated measures ANOVA test with 
stimulus (/ba/ vs. /wa/) x status (standard vs. 
deviant) x hemisphere (left vs. right) (F(l,9) = 14.3, 
p < .01). The /ba/ syllable elicited a larger Nlm than 
/wa/ regardless of being standard or deviant. This 
pattern was also observed in the American control. 
In the /ra-la/ experiment, the Japanese subjects 
showed double Nlm elicitation with a span of 
approximately 155 ms. The timing of the second 
Nlm matched that of the formant transitions in the 
stimuli after an initial 155 ms steady formant 
structure. The American control, however, did not 
exhibit a clear second Nlm response. Overall, the 
Japanese subjects showed slightly larger (not 
statistically significant) Nlm for both phonetic 
contrasts in the left hemisphere than the right 
whereas the American control demonstrated left- 
hemisphere do mi nance with the left-to-right 
magnitude ratios at 2.1 ~ 3.4. 
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Figure 3. Sample single-subject MEG waveforms 
from left and right hemispheres. Triangle marker 
indicates Nlm. (a): standard = /ba/, deviant = /wa/; 
(b) standard = /la/, deviant = /ra/. (thin line = 
standard, thick line = deviant; vertical scale = 100 
fT/cm, horizontal scale = 200 ms). 

Table 1(a) gives the numerical values of MMFs for 
the /ba-wa/ contrast. For the Japanese subjects, the 
MMF peak amplitudes in both hemispheres were 
significant compared with the noise level in a 
baseline from -100 ms to 0 ms (two-tailed t-test, p < 


.01). The American subject showed bigger mismatch 
in the left hemisphere and smaller responses in the 
right (outliers of 99% confidence interval for the 
Japanese group in both hemispheres). 

Table 1(b) shows the MMF corresponding to the 
initial Nlm for the /ra-la/ contrast. For Japanese 
subjects, the MMF activation exhibited two peaks in 
response to the double Nlm responses, which were 
also bilaterally significant (Two subjects did not 
show MMFs respectively in response to the first 
Nlm and second Nlm when /la/ was presented as 
standard and /ra/ as deviant). The American 
control’s MMFs were much larger to the /ra-la/ 
stimuli in the left hemisphere (outliers of 99% 
confidence interval) and slightly smaller in the right. 
Table 1 .MMFs for the /ba-wa/ and /ra-la/ contrasts, 
(mean ± S.E. for Japanese subjects; American control 
in curly bracket; “/A/-/B/” notation stands for 
subtraction of standard /B/ from deviant /A/.) 




MMF 

Hemisphere 

/ba/-/wa/ 

/wa/-/ba/ 

Latency 

Left 

248.7 ± 22.3 
{279.7} 

197.8 ±8.4 
{165.0} 

(ms) 

Right 

263.4 ± 19.3 
{Null} 

245.5 ± 14.0 
{293.8} 

Amplitude Left 

28.1 ±1.7 
{35.2} 

43.7 ±3.8 
{58.3} 

(fT/cm) 

Right 

32.5 ±2.3 
{Null} 

45.1 ±3.9 
{30.1} 

(b) 

MMF 

Hemisphere 

/ra/-/la/ 

/la/-/ra/ 

Latency 

Left 

194.8 ± 15.2 
{229.4} 

197.4 ± 13.4 
{241.5} 

(ms) 

Right 

203.4 ± 15.9 
{175.1} 

204.3 ± 13.9 
{193.2} 

Amplitude Left 

28.9 ±1.8 
{47.5} 

28.7 ±2.3 
{38.8} 

(fT/cm) 

Right 

25.7 ±3.3 
{21.1} 

27.7 ± 3.4 
{22.8} 


Repeated measures ANOVA tests comparing native 
vs. normative contrasts showed a significant main 
effect of stimulus for the Japanese subjects (p < .01). 
Post-hoc Scheffe tests revealed that the main 
contribution to this difference was the /wa/-/ba/ 
condition. In fact, the MMFs for the /ba/-/wa/ 
subtraction were at the same level as for the 
normative contrast. To reduce the effects of Nlm 
differences in /ba/ and /wa/ on the MMF peak, a 
second analysis using the same stimuli as deviant 
and standard from across presentation blocks was 
performed. This analysis resulted in balanced MMFs 








for the oddball /ba/ and /wa/ stimuli, which were 
significantly larger than the normative MMFs. 

4 Discussion 

The behavioral data convincingly demonstrate the 
effects of language experience on perceptual 
performance, as shown in previous studies. The 
reduced discriminability seen behaviorally for the 
/ra-la/ contrast in Japanese subjects was matched by 
the MMF responses, which were also reduced for 
this normative contrast. The different activation 
patterns for the Nlm and MMF to the third formant 
of /r-1/ between Japanese subjects and the American 
control indicate that linguistic experience can 
modify speech processing at the pre-attentive 
sensory level. The data suggest that the MMF can be 
regarded as a sensitive neural index of the effects of 
linguistic experience and show further that the MMF 
exhibits a gradient property reflecting the nature of 
the acoustic differences in stimuli and hemispheric 
involvement in phonetic processing. Behavioral 
training for Japanese subjects in which the third 
formant of /r-1/ stimuli is exaggerated may increase 
the brain's response to this contrast and improve 
their performance. 
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